METHOD FOR PROMOTING TEMPORAL RESOLUTION OF 
SEQUENTIAL IMAGES 



FIELD OF THE INVENTION 
5 [0001] This invention relates to a method for promoting temporal resolution of 
images, particularly to a method that can improve temporal resolution of sequential 
images by interpolating some images. 
BACKGROUND OF THE INVENTION 

[0002] In the era of multimedia computers and digital communication systems, 
10 the speed control technique of bit-stream for signal encoding and transmitting is 
often referred to digital signal processing (DSP) methods. And in processing digital 
video signals, sometimes, if not all the times, part of a train of visual frames would 
be deleted for one reason or another that might degrade the video temporal 
resolution when playing a video entertainment system for example. More 
15 specifically speaking, the visual frames would become discrete in the event of a 
temporal resolution as low as or even lower than 15 visual frames per second. 
[0003] For remedy, a temporal interpolation method — a method for 
interpolating more images between two images by way of motion estimation so as 
to raise the temporal resolution — is adopted so far though, likelihood of discrete 
20 viewing impression still exists if the interpolation images are too far in aberration 
with their side pictures however. 
SUMMARY OF THE INVENTION 

[0004] The primary object of this invention is to provide a method for 
promoting temporal resolution of sequential images. 
25 [0005] In order to realize above said object, this invention proposes a method 
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for interpolating predictable images in a sequential image train so as to heighten 
the temporal resolution thereof. The method is firstly to choose a first visual frame 
in a train of sequential images and spilt it into a plurality of visual blocks according 
to a predetermined value, then choose a second visual frame in the train of 
5 sequential images. The following step is to find a mostly resembling visual block 
in the second visual frame corresponding to each visual block of the first visual 
frame, then calculate the vector displacement of corresponding respective blocks in 
the visual frames. Now, the method is to rebuild a visual frame by utilizing the 
j~; vector displacement obtained in the previous step and estimate at a time point the 

rj= 10 pixel value of respective blocks of the first and the second visual frame, and 
combine them to create the rebuilding visual frame. Finally, such method is 

□ 

M. supposed to perform a median-value-filtering procedure then a spatial low-pass 

ill filtering procedure upon the rebuilt visual frame. 

iy [0006] For more detailed information regarding advantages or features of this 

15 invention, at least an example of preferred embodiment will be fully described 
below with reference to the annexed drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0007] The related drawings in connection with the detailed description of this 
invention to be made later are described briefly as follows, in which: 
20 Fig. 1 shows the flowchart of a method for promoting temporal 

resolution of sequential images of this invention; 

Fig. 2 is a schematic view showing the way of the method of this 
invention for calculating the vector displacement; and 

Figs. 3 A and 3B show the way to rebuild visual frames according to the 
25 method of this invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

[0008] As indicated in the flowchart of a method for promoting temporal 
resolution of sequential images of this invention, the method is firstly to select a 
first and a second visual frame of a train of sequential images, in which the first 
visual frame is split into a plurality of visual blocks according to a predetermined 
pixel value, which is generally equal to 16 pixels* 16 pixels. 

[0009] A first step 1 1 of the method is to provide pixel data to each visual block 
of the first and the second visual frame. A second step 12 is to search for a visual 
block in the second visual frame corresponding to each visual block in the first 
visual frame according to the input pixel value, and calculate the vector 
displacement between a pair of correspondent visual blocks in the first and the 
second visual frame respectively by way of motion estimation shown in Fig. 2. A 
third step 1 3 is to estimate the pixel value of each pair of respective corresponding 
visual blocks in the first and the second visual frame at a time point according to 
the vector displacement obtained in the previous step 12 and thereby rebuild a new 
visual frame. A step 14 is a motion assortment step for splitting the visual blocks of 
the rebuilt visual frame after motion compensation into a first and a second visual 
block set and for performing a median-value-filtering procedure to the first visual 
block set. The first visual block set is a visual block set with vector variation when 
a pair of corresponding visual blocks in the respective first and second visual frame 
is changed owing to image movement of an article in the visual frames. The second 
visual block set is a visual block set without vector variation of corresponding 
visual blocks in the first and the second visual frame, namely, a zero-vector- 
displacement visual block set. A step 15 is to perform a median- value-filtering 
procedure when a visual block set is found the first visual block set in step 14. A 



step 1 6 is to apply a spatial low-pass-filtering procedure to the first visual block set, 
which has undergone the median- value-filtering procedure in the previous step, 
and to the second visual block set that doesn't undergo the step 15. And, a step 17 
is to output the new visual frame rebuilt on the basis of the first and the second 
visual frame. 

[0010] The foregoing first visual frame might be the current visual frame and 
the second visual frame might be the previous visual frame. 

[0011] In Fig. 2, which is a schematic view showing the way of the method of 
this invention for calculating a vector displacement, a visual frame 20 is the current 
visual frame split into a plurality of visual blocks according to a predetermined 
value while another visual frame 25 is the previous visual frame. In order to 
calculate the vector displacement between respective visual blocks in the current 
and the previous visual frames 20, 25, a pair of corresponding visual blocks in 
those visual frames must be found and compared firstly. 

[0012] Figs. 3A and 3B show the way to rebuild visual frames according to the 
method of this invention. For rebuilding a visual frame, the method is to form 
firstly a first and a second interpolation visual frame, then try to secure the mean 
value of pixel data in the interpolation visual frames to be taken as pixel values of 
the new visual frame. 

[0013] In Fig. 3A — a schematic view of a method for forming a first 
interpolation visual frame — a block Bl located at position (x, y) in a first visual 
frame 30 has a vector displacement (dx, dy) with respect to a second visual frame 
35, namely, the block Bl at position (x+dx, y+dy) in the second visual frame 35, 
i.e., a block B2 in Fig. 3A. Now, assuming a block B3 in a first interpolation visual 
frame 33-A is located at position (x, y) while at a corresponding position (x+dx/2, 



y+dy/2) in the second visual frame 35, namely a block C3b in Fig. 3A, and at a 
corresponding position (x-dx/2, y-dy/2) in the first visual frame 30, namely a block 
C3f, then the block B3 may be rebuilt by using the mean value of pixel data in the 
block C3b and C3f. Accordingly, the first interpolation visual frame 33-A can be 
5 rebuilt in such a manner from block to block of the first and the second visual 
frames 30, 35. 

[0014] In Fig. 3B — a schematic view of a method for forming a second 
interpolation visual frame, a position (x+dx/2, y+dy/2) would rather be considered 
herein than a block rebuilding position (x, y). The position of the block Bl in a 

10 second interpolation visual frame 33-B, namely the position of the block B3, is 
predictable basing on its moving vector in the first visual frame 30. The pixel data 
of the block B3 is obtainable according to the mean value of that of the block Bl 
and the block B2 in the first and the second visual frame 30, 35 respectively. 
[0015] The method for creating the second interpolation visual frame is more 

15 logical with better correctness than that for the first one, however, the pixels 
therein may generate some rebuilding pixels overlapped or, on the contrary, lack 
some corresponding interpolation pixels for rebuilding to result in some blank 
sections in a rebuilt visual frame. For the former case with overlapped pixels, we 
suggest to calculate and find a mean value for substitution of the pixels, or use the 

20 method for forming the first interpolation visual frame for prediction of the pixel 
values. After the first and the second interpolation visual frame 33-A, 33-B are 
generated, the method is to calculate the mean value of pixel data in each block of 
the interpolation visual frames for rebuilding a visual frame, then heighten the 
image quality by taking advantage of the temporal information. 

25 [0016] Since the foregoing method for rebuilding visual frames is incapable of 



tackling rotation, enlargement, and shrinkage of an article in a train of sequential 
images effectively, consequently, application of temporal information by 
performing the median-filtering procedure upon a non-zero-vector-displacement 
rebuilding block is considered useful for eliminating the unnatural continuity of the 
5 sequential images when replaying. This procedure is to firstly compare the pixel 
data of corresponding blocks in the first visual frame, the second visual frame, and 
a rebuilt visual frame so as to take a pixel value of each block set compared as a 
median value, then perform a mean operation of the median value of each block 

z: pixel and the pixel value of each block of the rebuilt visual frame according to the 

■V 10 following formula: 

I"" NewY= 

Q 

H (MotComp + Median (MotCompY, PastY, Future Y))/2 

where NewY represents the result obtained from the mean operation of the median 
? value of each block pixel and the pixel value of each block of the rebuilt visual 

15 frame; MotCompY represents a rebuilt block produced by the foregoing method; 

PastY represents the block pixel value in the second visual frame; and FutureY 

represents the block pixel value of the first visual frame. 

[0017] The rebuilt visual frame processed by the median-value-filtering 
procedure is further processed with the spatial low-pass filtering procedure by any 
20 of the following formulas: 
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[0018] In the above described, at least one preferred embodiment has been 
described in detail with reference to the drawings annexed, and it is apparent that 
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numerous variations or modifications may be made without departing from the true 
spirit and scope thereof, as set forth in the claims below. 
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